Head up display

ABSTRACT

A head up display superimposes a virtual image on an actual view in accordance with a traveling status of a vehicle. Vehicle information containing a lean of the vehicle in a pitch direction is detected by a first posture sensor, and a lean in a roll direction is detected by a second posture sensor. A controller controls display of video on the basis of the vehicle information. A mirror reflects the video to project the video to a windshield or combiner. A mirror driver changes an angle and/or a position of the mirror. At least one of a display state of the virtual image in a display area for the video or the angle and/or the position of the mirror via the mirror driver is adjusted by the controller on the basis of the lean in the pitch direction and/or the lean in the roll direction.

TECHNICAL FIELD

The present invention relates to a technique of a head up display, andparticularly, the present invention relates to an effective techniquefor application to a head up display using AR (Augmented Reality).

BACKGROUND ART

For example, in a vehicle such as an automobile, information such asvehicle speed or the number of revolutions of an engine is normallydisplayed on an instrument panel board (or an instrument panel) in adashboard. Further, a screen for car navigation or the like is displayedon a display that is built in the dashboard or mounted on the dashboard.In a case where a driver visually recognizes these kinds of information,there is need to largely move a visual line. Therefore, as a techniqueto reduce a moving amount of the visual line, a head up display (Head UpDisplay: hereinafter, referred to also as an “HUD”) is known thatprojects information such as vehicle speed or information such as aninstruction related to car navigation onto a windshield or a combinerand displays the information.

An in-vehicle display apparatus including the HUD, a vehicle may vibrateor be inclined in accordance with a traveling status. Therefore, a casewhere a problem occurs on visibility of video to be displayed, and acase where appropriate content cannot be displayed may be generated.

As a technique related to improvement and the like of the visibility ofthe video to be displayed in the HUD, for example, Japanese PatentApplication Publication No. 2013-237320 (Patent document 1) describesthat a rotational component generated on a vehicle body is acquired as alean of the vehicle body, video is rotatively correctedthree-dimensionally on the basis of this, a position and the lean fordisplaying the rotatively corrected video are determined, and the videois projectively displayed.

Further, Japanese Patent Application Publication No. 2007-55365 (Patentdocument 2) describes that when a distance scale is displayed on an HUD,information on a traveling spot at which own vehicle is currentlytraveling and information on a traveling preset spot toward which theown vehicle is traveling are acquired from map data of a navigationapparatus, and an angle of gradient of a road on which the own vehicleis traveling is acquired on the basis of this, a display height from theground of the distance scale is corrected and displayed by using acorrection coefficient based on the angle of gradient.

Further, Japanese Patent Application Publication No. 2006-7867 (Patentdocument 3) describes that a display position of generated video iscontrolled in accordance with a detected traveling status such as rightor left turning, acceleration or deceleration. For example, in a casewhere left turning is detected, the display position is shifted to aleft direction. In a case where right turning is detected, the displayposition is shifted to a right direction.

Further, Japanese Patent Application Publication No. 2015-202842 (Patentdocument 4) describes that a display position of video information ismoved toward a direction in which a field of view of a driver is securedin accordance with a vehicle state.

RELATED ART DOCUMENTS Patent Documents

-   Patent document 1: Japanese Patent Application Publication No.    2013-237320-   Patent Document 2: Japanese Patent Application Publication No.    2007-55365-   Patent document 3: Japanese Patent Application Publication No.    2006-7867-   Patent Document 4: Japanese Patent Application Publication No.    2015-202842

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

The HUD is an apparatus that projects video onto a windshield or acombiner, whereby a driver is caused to recognize the video as a virtualimage in vehicle exterior. On the other hand, an HUD that realizes aso-called AR function (hereinafter, referred to also as “AR-HUD”), inwhich information related to an object and the like can be indicated tothe driver by displaying the virtual image so as to superimpose thevirtual image onto an actual scene (actual view) of the vehicle exteriorseen through the windshield or the combiner, is known. In such anAR-HUD, there is also need to execute adjustment for maintaining thevisibility of the video to be displayed, suitability, or the like inaccordance with a traveling status of the vehicle.

In this point, for example, by using the technique as described in anyof above Patent documents 1 to 3, even in a case where the vehiclevibrates or is inclined in accordance with the traveling status, it ispossible to reduce or solve an influence on visibility or suitability ofthe video to be displayed (virtual image). On the other hand, in thesetechniques, by using the video displayed in the display area for thevirtual image in the HUD as a target, the display position, the displaycontent or the like is adjusted in accordance with the traveling status.However, in case of the HUD, by taking into consideration the fact thatan effect to reduce a moving amount of the visual line of the driver canbe acquired regardless of the traveling status of the vehicle, it isdesirable to execute not only adjustment of the video in the displayarea, but also adjustment of the display area itself to move a positionthereof.

In this point, in the technique described in Patent document 4, thedisplay area itself in the HUD can be moved in accordance with thevehicle state. However, the technique described in Patent document 4 isintended to secure the field of view of the driver even in a case wherea change occurs in the vehicle state. The display area in the HUD ismoved to a position at which the display area does not constitute anobstacle for the driver. When such a technique is applied to the AR-HUD,a case where the virtual image cannot be superimposed onto the actualview included in the field of view of the driver occurs. Thus, the ARfunction does not have effectiveness.

Thus, it is an object of the present invention to provide a head updisplay capable of displaying a virtual image in accordance with atraveling status of a vehicle so as to appropriately superimpose thevirtual image onto an actual view.

The foregoing and other objects, and new features of the presentinvention will become more apparent from the detailed description of thepresent specification and the appending drawings.

Means for Solving the Problem

An outline of representative invention of the present inventiondisclosed in the present application will briefly be explained asfollows.

Ahead up display according to a representative embodiment of the presentinvention is a head up display for a vehicle, including: a vehicleinformation acquiring unit configured to acquire vehicle informationcontaining a lean of the vehicle in a front-back direction (pitchdirection), which is detected by a first posture sensor, and a lean in ahorizontal direction (roll direction), which is detected by a secondposture sensor, the first and second posture sensors being mounted onthe vehicle; a controller configured to control display of video on abasis of the vehicle information acquired by the vehicle informationacquiring unit; a video display configured to form the video on a basisof an instruction from the controller; a mirror configured to reflectthe video formed by the video display to project the video to awindshield or combiner; and a mirror driver configured to change anangle and/or a position of the mirror on a basis of an instruction fromthe controller.

Further, the controller is configured to adjust at least one of adisplay state of a virtual image in a display area for the video or theangle and/or the position of the mirror via the mirror driver on a basisof the lean in the front-back direction (pitch direction) and/or thelean in the horizontal direction (roll direction) in the vehicleinformation so that the virtual image is superimposed onto a scene andthe virtual image can be displayed to a driver.

Effects of the Invention

Effects obtained by representative invention of the present inventiondisclosed in the present application will briefly be explained asfollows.

Namely, according to the representative embodiment of the presentinvention, it becomes possible to display a virtual image in an AR-HUDin accordance with a traveling status of a vehicle so as toappropriately superimpose the virtual image onto an actual view.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating an outline of aconfiguration example of the whole head up display according to a firstembodiment of the present invention;

FIG. 2 is a view illustrating an outline of an example of an operationconcept of the head up display according to the first embodiment of thepresent invention;

FIG. 3 is a view illustrating an outline of an example of a hardwareconfiguration relating to acquisition of vehicle information accordingto the first embodiment of the present invention;

FIG. 4 is a functional block diagram illustrating details of aconfiguration example of the head up display according to the firstembodiment of the present invention;

FIG. 5 is a view illustrating details of an example of a configurationrelating to adjustment of a display distance according to the firstembodiment of the present invention;

FIG. 6 is a flowchart illustrating an outline of an example of aninitial operation according to the first embodiment of the presentinvention;

FIG. 7 is a flowchart illustrating an outline of an example of a normaloperation according to the first embodiment of the present invention;

FIG. 8 is a flowchart illustrating an outline of an example of abrightness level adjusting process according to the first embodiment ofthe present invention;

FIG. 9 is a view illustrating an outline of an example in which aposition of a display area for a virtual image is adjusted up and downaccording to the first embodiment of the present invention;

FIG. 10 is a flowchart illustrating an outline of an example of a mirroradjusting process according to the first embodiment of the presentinvention;

FIG. 11 is a flowchart illustrating an outline of an example of avibration correcting process according to the first embodiment of thepresent invention;

FIGS. 12(a) and 12(b) are views each illustrating an outline of anexample of adjustment of the display distance by using functional liquidcrystal films according to the first embodiment of the presentinvention;

FIG. 13 is a view illustrating an outline of a configuration example ofa diffuser plate composed of the functional liquid crystal filmsaccording to the first embodiment of the present invention;

FIG. 14 is a view illustrating an outline of an example of theadjustment of the display distance by using a plurality of mirrorsaccording to the first embodiment of the present invention;

FIG. 15 is a view illustrating an outline of an example of theadjustment of the display distance by using movable lenses according tothe first embodiment of the present invention;

FIGS. 16(a) and 16(b) are views each illustrating an outline of anexample of the adjustment of the display distance by using photochromicmirrors according to the first embodiment of the present invention;

FIG. 17 is a view illustrating an outline of a configuration example ofthe photochromic mirrors according to the first embodiment of thepresent invention;

FIG. 18 is a view illustrating an outline of an example of theadjustment of the display distance by using a movable diffuser plateaccording to the first embodiment of the present invention;

FIGS. 19(a) and 19(b) are views each illustrating an outline of anexample of the adjustment of the display distance by using a movableoptical filter according to the first embodiment of the presentinvention;

FIG. 20 is a view illustrating an outline of an example of theadjustment of the display distance by using a comb-shaped optical filteraccording to the first embodiment of the present invention;

FIG. 21 is a view illustrating an outline of an example of jolting of avehicle and an example of a display state of a virtual image in a normalhead up display;

FIG. 22 is a view illustrating an outline of an example of the joltingof the vehicle and an example of the display state of the virtual imagein the normal head up display;

FIG. 23 is a view illustrating an outline of an example of the joltingof the vehicle and an example of the display state of the virtual imagein the normal head up display;

FIGS. 24(a) and 24(b) are views each illustrating an outline of anexample in which a display position of a virtual image is adjusted upand down according to a second embodiment of the present invention;

FIGS. 25(a) to 25(c) are views each illustrating an outline of anexample in which the display position of the virtual image is adjustedup and down according to the second embodiment of the present invention;

FIGS. 26(a) and 26(b) are views each illustrating an outline of aconfiguration example of a mirror and a mirror driver according to thesecond embodiment of the present invention;

FIGS. 27(a) and 27(b) are views each illustrating an outline of anexample in which right and left leans of a display area for the virtualimage is adjusted according to the second embodiment of the presentinvention;

FIG. 28 is a view illustrating an outline of an example in which thevirtual image and a display position of the display area are adjusted upand down according to the second embodiment of the present invention;

FIG. 29 is a view illustrating an outline of an example of a virtualimage adjusting amount table according to the second embodiment of thepresent invention;

FIGS. 30(a) and 30(b) are views each illustrating an outline of anexample in which a degree or sensitivity of adjustment of a displaystate of the virtual image is allowed to be set according to the secondembodiment of the present invention;

FIG. 31 is a flowchart illustrating an outline of an example of a normaloperation according to the second embodiment of the present invention;

FIG. 32 is a flowchart illustrating an outline of an example of adisplay state adjusting process according to the second embodiment ofthe present invention; and

FIG. 33 is a flowchart illustrating an outline of an example of a mirrorstate adjusting process according to the second embodiment of thepresent invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described indetail with reference to the drawings. Note that the same referencenumeral is generally applied to the same element in all of the drawingsfor explaining the embodiments and its repeated explanation will beomitted. On the other hand, an element (or component) that has beenexplained in a drawing while applying a reference numeral thereto is notillustrated again when another drawing is to be explained, but theelement may be referred to by applying the same reference numeralthereto.

Note that in the following explanation, with respect to motiondirections of a vehicle, an axis of a direction in which the vehiclegoes straight ahead may be described as a rolling axis and a directionin which the vehicle rotates around the rolling axis may be described asa roll direction. Further, an axis that belongs to the same horizontalplane as the rolling axis and crosses at right angles with the rollingaxis may be described as a pitching axis, and a direction in which thevehicle rotates around the pitching axis may be described as a pitchdirection. Further, an axis that belongs to the same vertical plane asthe rolling axis and crosses at right angles with the rolling axis maybe described as a yawing axis, and a direction in which the vehiclerotates around the yawing axis may be described as a yaw direction.

First Embodiment

<Apparatus Configuration>

FIG. 2 is a view illustrating an outline of an example of an operationconcept of the head up display according to a first embodiment of thepresent invention. An AR-HUD 1 according to the present embodimentcauses a mirror 51 or a mirror 52 (for example, a free-form surfacemirror, a mirror having an asymmetric shape with respect to an opticalaxis, or the like) to reflect video displayed on a video display 30 toproject the video onto a windshield 3 of a vehicle 2. The video display30 is constructed by a projector or an LCD (Liquid Crystal Display).

A driver 5 views video projected onto the windshield 3, thereby visuallyrecognizing the video through the transparent windshield 3 in frontthereof as a virtual image. In the present embodiment, by adjusting anangle of the mirror 52 as will be described later, a position of thewindshield 3 at which video is to be projected is adjusted, whereby itis possible to adjust a display position of the virtual image viewed bythe driver 5 in a vertical direction. Namely, the mirror 52 allows anangle to be adjusted by being rotated around a rotation axis of ahorizontal direction.

Further, in the present embodiment, by using various kinds of methods(will be described later), a display distance of the virtual image canbe adjusted so that the virtual image is displayed at a nearby position(for example, 2 to 3 m ahead) or a faraway position (for example, 30 to40 m ahead). Then, by adjusting the display position and/or the displaydistance of the virtual image so as to superimpose the virtual imageonto an actual view of vehicle exterior (a road, a building, a person orthe like), an AR function is realized.

FIG. 1 is a functional block diagram illustrating an outline of aconfiguration example of the whole head up display according to thefirst embodiment of the present invention. The AR-HUD 1 mounted on thevehicle 2 includes a vehicle information acquiring unit 10, a controller20, a video display 30, a display distance adjusting mechanism 40, amirror driver 50, the mirror 52, and a speaker 60, for example. In theexample of FIG. 1, a shape of the vehicle 2 is displayed like apassenger vehicle. However, the vehicle 2 is not limited to this, andthe AR-HUD 1 can be applied to various kinds of vehicles appropriately.

The vehicle information acquiring unit 10 is composed of informationacquiring devices such as various kinds of sensors mounted on respectiveportions of the vehicle 2 as will be described later. The informationacquiring devices detect various kinds of events that occur in thevehicle 2, and detect and/or acquire values of various kinds ofparameters related to a traveling status at predetermined intervals,thereby acquiring and outputting vehicle information 4. For example, asillustrated in FIG. 1, speed information and gear information of thevehicle 2, handle steering angle information, lamp lighting information,outside light information, distance information, infrared information,engine ON/OFF information, camera video information (vehicleinterior/vehicle exterior), acceleration/posture information, GPS(Global Positioning System) information, navigation information,vehicle-to-vehicle communication information, road-to-vehiclecommunication information, and the like may be contained in the vehicleinformation 4.

The controller 20 has a function configured to control an operation ofthe AR-HUD 1. For example, a CPU (Central Processing Unit) and softwareto be executed by the CPU are implemented in the controller 20. Thecontroller 20 may be configured by hardware such as a microcomputer oran FPGA (Field Programmable Gate Array). As also illustrated in FIG. 2,the controller 20 drives the video display 30 to form video to bedisplayed as a virtual image on the basis of the vehicle information 4acquired by the vehicle information acquiring unit 10 and the like. Thecontroller 20 causes the video to be reflected appropriately by means ofthe mirror 52 or the like, thereby projecting the video onto thewindshield 3. Further, by methods as will be described later, thecontroller 20 executes controls to adjust a display position of adisplay area for the virtual image, and adjust the display distance ofthe virtual image.

As described above, the video display 30 is a device configured by aprojector or an LCD, for example. The video display 30 forms video fordisplaying the virtual image on the basis of an instruction from thecontroller 20 to project or display the video. The display distanceadjusting mechanism 40 is a mechanism for adjusting a distance from thedriver 5 to the displayed virtual image on the basis of an instructionfrom the controller 20, for example. The display distance adjustingmechanism 40 implements any one or more of various kinds of methods ofadjusting the display distance as will be described later.

The mirror driver 50 adjusts an angle of the mirror 52 on the basis ofan instruction from the controller 20 to adjust a position of thedisplay area for the virtual image in a vertical direction. Adjustmentof the position of the display area in the virtual image will bedescribed later. The speaker 60 executes an audio output related to theAR-HUD 1. For example, the speaker 60 can execute audio guidance of anavigation system, an audio output when the driver 5 is notified of awarning or the like by the AR function, and the like.

FIG. 3 is a view illustrating an outline of an example of a hardwareconfiguration relating to acquisition of the vehicle information 4 inthe head up display according to the present embodiment. Here, thehardware configuration of a part of the vehicle information acquiringunit 10 and the controller 20 is mainly illustrated in FIG. 3.Acquisition of the vehicle information 4 is executed by the informationacquiring devices such as various kinds of sensors connected to an ECU(Electronic Control Unit) 21 under controls of the ECU 21, for example.

As these information acquiring devices, for example, there are devicessuch as a vehicle speed sensor 101, a shift position sensor 102, ahandle steering angle sensor 103, a headlight sensor 104, an illuminancesensor 105, a chromaticity sensor 106, a distance measuring sensor 107,an infrared sensor 108, an engine start sensor 109, an accelerationsensor 110, an posture sensor 111, a temperature sensor 112, aroad-to-vehicle communication wireless receiver 113, avehicle-to-vehicle communication wireless receiver 114, a camera(vehicle interior) 115, a camera (vehicle exterior) 116, a GPS receiver117, and a VICS (Vehicle Information and Communication System,registered trademark (the same applies hereinafter)) receiver 118. Allof these devices are not always provided. Further, other kinds ofdevices may be provided. It is possible to appropriately use the vehicleinformation 4 that can be acquired by the provided devices.

The vehicle speed sensor 101 acquires the speed information of thevehicle 2. The shift position sensor 102 acquires current gearinformation of the vehicle 2. The handle steering angle sensor 103acquires handle steering angle information. The headlight sensor 104acquires lamp lighting information related to ON/OFF of a headlight. Theilluminance sensor 105 and the chromaticity sensor 106 acquires outsidelight information. The distance measuring sensor 107 acquires distanceinformation on a distance between the vehicle 2 and an external object.The infrared sensor 108 acquires infrared information related topresence or absence of an object and a distance therefrom in a shortrange from the vehicle 2. The engine start sensor 109 detects engineON/OFF information.

The acceleration sensor 110 and the posture sensor 111 acquiresacceleration/posture information composed of acceleration and angularvelocity as information on posture and behavior of the vehicle 2. Thetemperature sensor 112 acquires temperature information on temperatureinside and outside the vehicle 2. The road-to-vehicle communicationwireless receiver 113 and the vehicle-to-vehicle communication wirelessreceiver 114 respectively acquire the road-to-vehicle communicationinformation received by means of road-to-vehicle communication betweenthe vehicle 2 and a road, a sign, a signal or the like and thevehicle-to-vehicle communication information received by means ofvehicle-to-vehicle communication between the vehicle 2 and each of othersurrounding vehicles.

The camera (vehicle interior) 115 and the camera (vehicle exterior) 116respectively acquire camera video information (vehicle interior/vehicleexterior) by photographing a dynamic image of statuses of the vehicleinterior and the vehicle exterior. The camera (vehicle interior) 115photographs posture of the driver 5, a position and motion of eyes, andthe like, for example. By analyzing the acquired dynamic image, it ispossible to grasp a tiredness status, a position of a visual line andthe like of the driver 5, for example. Further, the camera (vehicleexterior) 116 photographs a status of surroundings such as ahead or backof the vehicle 2. By analyzing the acquired dynamic image, it ispossible to grasp presence or absence of a moving object such as othervehicle or a person of the surroundings, a building or a landform, aroad surface status (rain, snow, freeze, unevenness, and the like), forexample.

The GPS receiver 117 and the VICS receiver 118 respectively acquire GPSinformation acquired by receiving a GPS signal and VICS informationacquired by receiving a VICS signal. They may be implemented as a partof a car navigation system that acquires and uses these kinds ofinformation.

FIG. 4 is a functional block diagram illustrating details of aconfiguration example of the head up display according to the presentembodiment. As an example, FIG. 4 illustrates a case where the videodisplay 30 is a projector. The video display 30 includes respectiveunits such as a light source 31, an illumination optical system 32, or adisplay element 33, for example. The light source 31 is a member thatgenerates illumination light for projection. For example, ahigh-pressure mercury lamp, a xenon lamp, an LED (Light Emitting Diode)light source, a laser light source and the like can be utilized. Theillumination optical system 32 is an optical system that concentratesthe illumination light generated by the light source 31, and furtherequalizes the illumination light to irradiate the illumination light tothe display element 33. The display element 33 is an element thatgenerates video to be projected. For example, a transmissive liquidcrystal panel, a reflective liquid crystal panel, a DMD (DigitalMicromirror Device) (registered trademark) panel and the like can beutilized.

More specifically, the controller 20 includes respective units such asthe ECU 21, an audio output unit 22, a nonvolatile memory 23, a memory24, a light source adjusting unit 25, a distortion correcting unit 26, adisplay element driver 27, a display distance adjusting unit 28, and amirror adjusting unit 29. As illustrated in FIG. 3, the ECU 21 acquiresthe vehicle information 4 via the vehicle information acquiring unit 10,and records, stores or reads out the acquired information in thenonvolatile memory 23 or the memory 24 as necessary. Setting informationsuch as various kinds of set values or parameters for control may bestored in the nonvolatile memory 23. Further, the ECU 21 executes adedicated program to generate video data related to the virtual image tobe displayed as the AR-HUD 1.

The audio output unit 22 outputs audio information via the speaker 60 asneeded. The light source adjusting unit 25 adjusts an amount ofluminescence of the light source 31 for the video display 30. In a casewhere there is a plurality of light sources 31, each of them may becontrolled separately. In a case where the video display 30 projects thevideo generated by the ECU 21 to the windshield 3 of the vehicle 2, thedistortion correcting unit 26 corrects distortion of the video, whichoccurs due to curvature of the windshield 3, by image processing. Thedisplay element driver 27 sends a driving signal based on the video dataafter correction by the distortion correcting unit 26 to the displayelement 33, and causes the display element 33 to generate video to beprojected.

In a case where there is need to adjust the display distance of thevirtual image, the display distance adjusting unit 28 drives the displaydistance adjusting mechanism 40 to adjust the display distance of thevideo projected from the video display 30. Various kinds of methods ofadjusting the display distance of the virtual image will be describedlater. In a case where there is need to adjust the position of thedisplay area itself of the virtual image, the mirror adjusting unit 29changes the angle of the mirror 52 via the mirror driver 50 to move thedisplay area for the virtual image up and down. Positioning of thedisplay area for the virtual image will be described later.

FIG. 5 is a view illustrating details of an example of a configurationrelating to adjustment of the display distance in the head up displayaccording to the present embodiment. The display distance adjusting unit28 of the controller 20 further includes a functional liquid crystalfilm ON/OFF controller 281, a lens moving unit 282, a photochromicmirror ON/OFF controller 283, a diffuser plate moving unit 284, anoptical filter moving unit 285, and the like as respective units each ofwhich is separately controlled by the ECU 21, for example. Further, ashardware or devices that are controlled or driven by the respectiveunits, the display distance adjusting mechanism 40 further includes afunctional liquid crystal film 401, a lens moving mechanism 402, aphotochromic mirror 403, a diffuser plate moving mechanism 404, anoptical filter moving mechanism 405, and the like. Methods of adjustingthe display distance of the virtual image by the respective units willbe described later.

Note that the AR-HUD 1 does not necessarily include all of these unitsand devices, and the AR-HUD 1 may appropriately include units requiredto implement one to be applied among the methods of adjusting thedisplay distance of the virtual image (will be described later).

<Processing Content>

FIG. 6 is a flowchart illustrating an outline of an example of aninitial operation by the head up display according to the presentembodiment. When a power source for the AR-HUD 1 is turned ON by turningan ignition switch ON in the stopped vehicle 2 (S01), the AR-HUD 1 firstacquires vehicle information via the vehicle information acquiring unit10 on the basis of an instruction from the controller 20 (S02). Thecontroller 20 then calculates an appropriate brightness level on thebasis of the outside light information, which is acquired by theilluminance sensor 105 and the chromaticity sensor 106, of the vehicleinformation 4 (S03). The controller 20 controls the light sourceadjusting unit 25 to set the amount of luminescence of the light source31 so as to become the calculated brightness level (S04). For example,in a case where the outside light is bright, the brightness level is setto be high. In a case where the outside light is dark, the brightnesslevel is set to be low.

The ECU 21 then determines and generates video to be displayed as avirtual image (for example, an initial image) (S05). After thedistortion correcting unit 26 executes a process to correct distortionwith respect to the generated video (S06), the display element driver 27drives and controls the display element 33 to form video to be projected(S07). Herewith, the video is projected onto the windshield 3, and thedriver 5 is allowed to visually recognize the virtual image. The ECU 21or the display distance adjusting unit 28 then calculates and determinesa display distance of the virtual image (S08). The display distanceadjusting unit 28 drives the display distance adjusting mechanism 40 tocontrol the display distance of the video projected from the videodisplay 30 (S09).

When activation or start of each of the units including the series ofinitial operation described above is completed in the whole AR-HUD 1, anHUD-ON signal is outputted. The controller 20 determines whether thissignal is received or not (S11). In a case where it is determined thatthis signal is not received, the controller 20 further waits for theHUD-ON signal for a fixed time (S12), and repeats a waiting process forthe HUD-ON signal (S12) until it is determined at Step S11 that theHUD-ON signal is received. In a case where it is determined at Step S11that the HUD-ON signal is received, the controller 20 starts a normaloperation for the AR-HUD 1 (will be described later) (S13), andterminates the series of initial operation.

FIG. 7 is a flowchart illustrating an outline of an example of thenormal operation by the head up display according to the presentembodiment. A basic processing flow in the normal operation is alsosubstantially similar to that in the initial operation illustrated inFIG. 6 described above. The AR-HUD 1 first acquires vehicle informationvia the vehicle information acquiring unit 10 on the basis of aninstruction from the controller 20 (S21). The controller 20 thenexecutes a brightness level adjusting process on the basis of theoutside light information, which is acquired by the illuminance sensor105 and the chromaticity sensor 106, of the vehicle information 4 (S22).

FIG. 8 is a flowchart illustrating an outline of an example of thebrightness level adjusting process by the head up display according tothe present embodiment. When the brightness level adjusting process isstarted, the controller 20 first calculates an appropriate brightnesslevel on the basis of the acquired outside light information (S221). Thecontroller 20 determines whether a change in a brightness level isrequired or not by comparing the appropriate brightness level with abrightness level that is set currently (S222). In a case where thechange is not required, the controller 20 terminates the brightnesslevel adjusting process as it is. On the other hand, in a case where thechange is required, the controller 20 controls the light sourceadjusting unit 25 to set the amount of luminescence of the light source31 so as to become the brightness level after the change (S223), andterminates the brightness level adjusting process. Note that even in acase where there is a difference between the appropriate brightnesslevel calculated at Step S221 and the brightness level currently set, itmay be determined at Step S222 that the change of the brightness levelis required only when the difference is equal to or more than apredetermined threshold value.

Returning to FIG. 7, the ECU 21 then changes, as needed, the video to bedisplayed as the virtual image from the current one on the basis of thelatest vehicle information 4 acquired at Step S21, determines andgenerates video after change (S23). Note that there may be a largenumber of patterns to change display content on the basis of the vehicleinformation 4 in accordance with content of the acquired vehicleinformation 4 and their combination. For example, there may be variouspatterns such as a case where the speed information is changed and anumeral value of speed display, which is always displayed, is therebychanged, or a case where an arrow figure for guidance is displayed ordeleted on the basis of the navigation information, or a shape and/or adisplay position of an arrow is changed.

In the present embodiment, the ECU 21 then executes an adjusting and/orcorrecting process in accordance with the traveling status of thevehicle 2 in order to maintain visibility and suitability of the displaycontent. In a case where there is need to adjust the position of thedisplay area for the virtual image itself, the ECU 21 first executes amirror adjusting process to change the angle of the mirror 52 via themirror driver 50 to move the display area for the virtual image up anddown (S24). Then, the ECU 21 further executes a vibration correctingprocess to correct the display position of the video in the display areawith respect to vibration of the vehicle 2 (S25). Detailed content ofthe adjusting and/or correcting process at Steps S24 and S25 will bedescribed later.

Then, after the distortion correcting unit 26 executes the process tocorrect distortion for the adjusted and/or corrected video (S26), thedisplay element driver 27 drives and controls the display element 33 toform video to be projected (S27). The ECU 21 or the display distanceadjusting unit 28 then calculates and determines the display distance ofthe virtual image (S28), and the display distance adjusting unit 28drives the display distance adjusting mechanism 40 to control thedisplay distance of the video projected from the video display 30 (S29).

When the power source is turned OFF due to stoppage or the like of thevehicle 2 during execution of the series of normal operation describedabove, an HUD-OFF signal is outputted to the AR-HUD 1. The controller 20determines whether this signal is received or not (S30). In a case whereit is determined that the HUD-OFF signal is not received, the processingflow returns to Step S21, and the controller 20 repeats the series ofnormal operation until it is determined that the HUD-OFF signal isreceived. In a case where it is determined that the HUD-OFF signal isreceived, the controller 20 terminates the series of normal operation.

<Mirror Adjusting Process>

FIG. 9 is a view illustrating an outline of an example in which theposition of the display area for the virtual image is adjusted up anddown in the head up display according to the present embodiment. Forexample, in each of left, central, and right views, an upper stageschematically illustrates a state of a status of a gradient of a road onwhich the vehicle 2 is traveling and a status of a viewing field of thedriver 5 when viewed from a side surface thereof. Further, a lower stageschematically illustrates, in each of the states, a status of an actualview in front of the vehicle exterior, which is viewed by the driver 5,and a position of a display area 6 (a rectangle indicated by a dashedframe) of the virtual image displayed so as to be superimposed onto theactual view.

A view at a left side illustrates a case where a gradient (an upwarddirection) of a road at a current location of the vehicle 2 issubstantially the same as a gradient (upward direction) of a road aheadas illustrated in the view of the upper stage, that is, a case where thevehicle 2 substantially travels on a flat road. In this case, asillustrated in the view of the lower stage, in order to superimpose thevirtual image (in the example of FIG. 9, a mark or an image of anexclamation point) onto the actual view in front of the vehicle exterior(in the example of FIG. 9, a vehicle ahead that travels on the road) bythe AR function and display it, the position of the display area 6 forthe virtual image in a vertical direction may be a normal position.Namely, the position of the display area 6 illustrated in the view ofthe lower stage at the left side becomes a basic display position of thedisplay area 6 in the vertical direction.

On the other hand, a view at a central portion illustrates a case wherea gradient (an upward direction) of a road at a current location of thevehicle 2 is larger than a gradient of the road ahead (the upwarddirection), that is, a case where the vehicle 2 travels on a road inwhich the front is a downhill. In this case, as illustrated in the viewof the upper stage, there is need to move the viewing field in adownward direction (a dotted frame in FIG. 9) in order to cause the roadahead to come into the viewing field with respect to a height of theviewing field of the driver 5 (a solid frame in FIG. 9) based on thegradient at the position of the vehicle 2. In this case, as illustratedin the view of the lower stage, the virtual image cannot be superimposedonto the actual view in front of the vehicle exterior by the AR functionin a state that the display position of the display area 6 for thevirtual image is the basic display position (a rectangle indicated bythe dotted frame). Therefore, there is need to move the display area 6itself in the downward direction in order to superimpose the displayarea 6 and display the virtual image.

Similarly, a view at a right side illustrates a case where a gradient(the upward direction) of a road at a current location of the vehicle 2is smaller than a gradient of the road ahead (the upward direction),that is, a case where the vehicle 2 travels on a road in which the frontis an uphill. In this case, as illustrated in the view of the upperstage, there is need to move the viewing field in an upward direction(the dotted frame in FIG. 9) in order to cause the road ahead to comeinto the viewing field with respect to the height of the viewing fieldof the driver 5 (the solid frame in FIG. 9) based on the gradient at theposition of the vehicle 2. In this case, as illustrated in the view ofthe lower stage, the virtual image cannot be superimposed onto theactual view in front of the vehicle exterior by the AR function in astate that the display position of the display area 6 for the virtualimage is the basic display position (the rectangle indicated by thedotted frame). Therefore, there is need to move the display area 6itself in the upward direction in order to superimpose the display area6 and display the virtual image.

Thus, the status that it is necessary to move the position of thedisplay area 6 for the virtual image in the vertical direction inaccordance with the traveling status is not limited to the case wherethere is a difference of a constant amount or more between the gradientof the current position and the gradient of the road ahead as theexample illustrated in FIG. 9. For example, in a case where speed of thevehicle 2 becomes higher on a highway or the like, the visual line ofthe driver 5 generally becomes far away compared with a normal state,and the height of the viewing field moves in the upward direction.Therefore, for example, in order to superimpose the virtual image ontothe actual view of vehicle exterior including other vehicles and thelike that exists further ahead compared with the normal state, there maybe need to move the display area 6 in the upward direction. When postureof a position of the driver 5 is changed while the vehicle 2 istraveling, a height position of the eyes of the driver 5 itself ischanged. A case where this causes the height of the viewing field tomove in the vertical direction is similar.

In the present embodiment, in the mirror adjusting process at Step S24illustrated in FIG. 7 described above, the mirror driver 50 controls theangle of the mirror 52 in accordance with the traveling status of thevehicle 2, the position of the display area for the virtual image in thevertical direction is adjusted as illustrated in the example of FIG. 9.

FIG. 10 is a flowchart illustrating an outline of an example of themirror adjusting process at Step S24 of FIG. 7. When the mirroradjusting process is started, the ECU 21 first acquires a current angleof the mirror 52 (S241), and further acquires current values ofparameters related to adjustment of the angle of the mirror 52 (that is,adjustment of the display position of the display area for the virtualimage) on the basis of the vehicle information 4 (S242).

Kinds of parameters to be required may differentiate depending on underwhat condition the display position of the display area is to beadjusted. For example, in the example illustrated in FIG. 9, as therelated parameter value, the ECU 21 acquires a value indicating adifference (a relative gradient) between the gradient of the currentposition of the vehicle 2 and the gradient of the road ahead. Forexample, it is possible to grasp the gradient of the current positionfrom the information on a lean of the vehicle 2 acquired by theacceleration/posture information. Further, by analyzing the camera videoinformation of the vehicle exterior, it is possible to grasp thegradient of the road ahead. Further, it is possible to acquire thegradient of the current position and the gradient of the road ahead onthe basis of three-dimensional road/landform information acquired fromthe navigation information or the like.

Subsequently, the ECU 21 calculates a target angle of the mirror 52 onthe basis of the parameter value acquired at Step S242, and standardsand conditions defined in advance (S243). By what logic and on the basisof what parameter the target angle is calculated may differentiatedepending on in what condition the display position of the display areais to be adjusted. For example, in the example illustrated in FIG. 9, ina case where the absolute value of the relative gradient between thecurrent location and the road ahead is equal to or more than apredetermined threshold value, the ECU 21 determines the target angle ofthe mirror 52 in accordance with a reference numeral of the relativegradient. As the predetermined threshold value described above, 1/x (“x”is a predetermined value) or the like of FOV (Field Of View: a viewingangle) of the display area for the virtual image in the verticaldirection can be used, for example.

Note that in the present embodiment, the target angle of the mirror 52is calculated on the basis of the current parameter value acquired atStep S242. However, the ECU 21 may estimate a near future status on thebasis of information regarding the current parameter value and a historyof past values, and calculate the target angle on the basis of anestimation result. For example, the ECU 21 may analyze a tendency oftransition of the value on the basis of a past history of the parametervalue, and estimate a near future parameter value on the basis of thetendency. Further, by analyzing the camera video information in front ofthe vehicle exterior, it is possible to estimate a peripheral status ofthe vehicle 2 in the near future, and grasp a status of the road infront of the vehicle 2 on the basis of the navigation information.

Subsequently, the ECU 21 determines whether there is a differencebetween the current angle of the mirror 52 acquired at Step S241 and atarget angle of the mirror 52 acquired at Step S243 or not (S244). Withrespect to this determination, for example, the ECU 21 may determinethat there is a difference in a case where the difference is equal to ormore than a predetermined threshold value. Further, the ECU 21 maydetermine that there is no difference in a case where the difference isless than the threshold value. Further, the ECU 21 may determine thatthere is a difference only in a case where a state that there is thedifference continues for a fixed time or longer. This makes it possibleto exclude an event in which a lean of the vehicle 2 changes temporarilyand instantaneously, such as a case where the vehicle 2 rides on a leveldifference of curbstone or the like, from targets to adjust the mirror52, for example.

In a case where it is determined at Step S244 that there is nodifference between the angles, the ECU 21 terminates the mirroradjusting process as it is. Namely, the ECU 21 does not adjust the angleof the mirror 52, and maintains a current angle thereof. On the otherhand, in a case where it is determined that there is a differencebetween the angles, the ECU 21 rotates the mirror 52 in a specifieddirection so as to become the target angle (S245). Specifically, the ECU21 outputs, to the mirror driver 50, a mirror adjusting signal to rotatethe mirror 52. The ECU 21 then determines whether the angle of themirror 52 reaches the target angle or not (S246). In a case where it isdetermined that the angle does not reach the target angle, theprocessing flow returns to Step S245, and the ECU 21 causes the mirrordriver 50 to continue to rotate the mirror 52. Namely, the ECU 21continues to output the mirror adjusting signal to the mirror driver 50.On the other hand, in a case where it is determined that the angle ofthe mirror 52 reaches the target angle, the ECU 21 stops the rotation ofthe mirror 52 (S247). Namely, the ECU 21 stops outputting the mirroradjusting signal to the mirror driver 50. The ECU 21 then terminates aseries of the mirror adjusting process.

<Vibration Correcting Process>

FIG. 11 is a flowchart illustrating an outline of an example of thevibration correcting process at Step S25 of FIG. 7. When the vibrationcorrecting process is started, the ECU 21 first acquires information ona vibration amount of the vehicle 2 on the basis of the vehicleinformation 4 (S251). For example, it is possible to grasp the vibrationamount (that is, an amount of vertical motion of the vehicle 2 with ashort cycle) on the basis of the acceleration/posture information, thecamera video information of the vehicle exterior and the like. In thepresent embodiment, the information on vibration is acquired on thebasis of the current vehicle information 4. However, the ECU 21 mayestimate a road surface status of surroundings of the vehicle 2 in thenear future by analyzing the camera video information in front of thevehicle exterior, for example, and estimate the vibration amount of thevehicle 2 in the near future on the basis of the road surface status.

The ECU 21 then determines whether the vibration amount acquired at StepS251 is equal to or more than a predetermined threshold value or not(S252). In a case where it is determined that the vibration amount isless than the threshold value, the ECU 21 terminates the vibrationcorrecting process as it is because the vibration is minute. Namely, theECU 21 does not execute correction of video to be displayed associatedwith vibration. On the other hand, in a case where it is determined thatthe vibration amount is equal to or more than the threshold value, theECU 21 calculates a display shift amount of the video in the displayarea (S253). For example, the ECU 21 calculates the display shift amountof the video in the display area from the vibration amount of thevehicle 2 on the basis of a ration of an actual height of the vehicle 2and a height of the display area for the virtual image. The ECU 21 thenoffsets a display position of the video in the display area up and downon the basis of the calculated display shift amount (S254), and the ECU21 terminates the series of vibration correcting process.

<Adjustment of Display distance of Virtual Image>

In a case where there is need to adjust the display distance of thevirtual image, the display distance adjusting unit 28 of the controller20 drives the display distance adjusting mechanism 40 to adjust thedisplay distance of the video to be projected from the video display 30.Hereinafter, a method of adjusting the display distance of the virtualimage by each of the display distance adjusting unit 28 and the displaydistance adjusting mechanism 40 illustrated in FIG. 5 will be described.

[Functional Liquid Crystal Film]

FIG. 12 is a view illustrating an outline of an example of adjustment ofthe display distance by using the functional liquid crystal film 401 inthe head up display according to the present embodiment. In the exampleof FIG. 12, a plurality of functional liquid crystal films 401 are usedas a diffuser plate (or a diffuser) 41 a. As illustrated in FIGS. 12(a)and 12(b), by changing a portion to be set to a white state for everyarea of each of the functional liquid crystal films 401 to change afocal distance for every area, the display distance of the virtual image(that is, a distance between the position of the eyes of the driver 5and the display position of the virtual image) is changed.

FIG. 13 is a view illustrating an outline of a configuration example ofthe diffuser plate 41 a by the functional liquid crystal films 401. Eachof the functional liquid crystal films 401 is a film that can control atransparent state and a white state by electricity. A portion of thewhite state in each of the functional liquid crystal films 401 serves asa diffuser plate, and video projected by the projector 30 a forms animage at this portion of the white state. In the present embodiment,with respect to each of the plurality of functional liquid crystal films401, a plurality of areas is separately controlled so as to become awhite state.

Returning to FIG. 12, in the configuration as illustrated in FIG. 12,the display position of the virtual image based on video projected fromthe projector 30 a is determined in accordance with the portion of thewhite state in each of the functional liquid crystal films 401 and adistance with a lens 42 a. Therefore, the plurality of functional liquidcrystal films 401 is arranged so that the distances with the lens 42 aare different from each other, and the functional liquid crystal filmON/OFF controller 281 illustrated in FIG. 5 sets any one of thefunctional liquid crystal films 401 to the white state for every areawith respect to the video projected from the projector 30 a. This makesit possible to change the display distance of the virtual image for eacharea.

Specifically, for example, as illustrated in FIG. 12(a), with respect toa target area (for example, an uppermost portion), only the functionalliquid crystal film 401 arranged at the nearest position to the lens 42a is set to the white state, and the other films 401 are set to atransparent state, whereby it is possible to set the display distance ofthe corresponding virtual image to the nearest distance. Conversely, asillustrated in FIG. 12(b), with respect to the target area (for example,the uppermost portion), only the functional liquid crystal film 401arranged at the farthest position from the lens 42 a is set to the whitestate, and the other films 401 are set to the transparent state, wherebyit is possible to set the display distance of the corresponding virtualimage to the farthest distance.

In the examples of FIG. 12 and FIG. 13, a case where three areas for thevideo to be displayed are provided in the vertical direction isdescribed as an example. However, the number of areas including exampleswhich will be explained below is not limited to this. Further, adividing direction is not limited to the vertical direction, and areascan of course be divided in the horizontal direction. Further, thenumber of functional liquid crystal films 401 is also not limited tothree pieces as illustrated in FIGS. 12 and 13, and can be setappropriately in accordance with the number of areas.

[Arrangement of Plural of Mirrors]

FIG. 14 is a view illustrating an outline of an example of theadjustment of the display distance by using a plurality of mirrors inthe head up display according to the present embodiment. In the exampleof FIG. 14, a plurality of mirrors 51 a is arranged between an LCD 30 band the lens 42 a as illustrated in FIG. 14. Video from the LCD 30 b isreflect by different mirrors 51 a for every area, and is made incidentto the lens 42 a. This makes it possible to differentiate distances fromthe LCD 30 b to the lens 42 a for each area, and to change the displaydistance of the virtual image in accordance with this.

Specifically, for example, as illustrated in FIG. 14, by displaying thevideo from the LCD 30 b at the area reflected by the mirror 51 aarranged at the farthest position from the LCD 30 b (also, at thefarthest position from the lens 42 a), it is possible to set the displaydistance of the corresponding virtual image to the farthest distance.Conversely, by displaying the video from the LCD 30 b at the areareflected by the mirror 51 a arranged at the nearest position from theLCD 30 b (also, at the nearest position from the lens 42 a), it ispossible to set the display distance of the corresponding virtual imageto the nearest distance.

Note that in the example of FIG. 14, the number of mirror 51 a is alsonot limited to three pieces as illustrated in FIG. 14, and can be setappropriately in accordance with the number of areas.

[Movable Lens]

FIG. 15 is a view illustrating an outline of an example of theadjustment of the display distance by using movable lenses in the headup display according to the present embodiment. In the example of FIG.15, after video projected from the projector 30 a is formed on adiffuser plate (or diffuser) 41 b, the video is made incident to themirror 52 via movable lenses 42 b provided so as to be divided into aplurality of areas.

Here, it is possible to separately move each of the movable lenses 42 balong an optical axis direction by the lens moving unit 282 and the lensmoving mechanism 402 illustrated in FIG. 5. The display position of thevirtual image based on the video projected from the projector 30 a isdetermined in accordance with the distance between the diffuser plate 41b and each of the movable lenses 42 b. Therefore, by moving the movablelenses 42 b to change a focal distance for every area, it is possible tochange the display distance of the virtual image.

Specifically, for example, as illustrated in FIG. 15, by moving themovable lens 42 b to the nearest position to the diffuser plate 41 blike the area at an uppermost portion, it is possible to set the displaydistance of the corresponding virtual image to the nearest position.Conversely, by moving the movable lens 42 b to the farthest positionfrom the diffuser plate 41 b like the area at a lowermost portion, it ispossible to set the display distance of the corresponding virtual imageto the farthest distance.

Note that in the example of FIG. 15, the number of movable lenses 42 bis also not limited to three illustrated in FIG. 15, and can be setappropriately in accordance with the number of areas.

[Photochromic Mirror]

FIG. 16 is a view illustrating an outline of an example of theadjustment of the display distance by using a photochromic mirror 51 bin the head up display according to the present embodiment. In theexample of FIG. 16, a plurality of photochromic mirrors 403 is arrangedbetween the LCD 30 b and the lens 42 a as illustrated in FIG. 16 so asto constitute the photochromic mirror 51 b by arranging the photochromicmirrors 403 to become a matrix shape when viewed from across sectionaldirection. As illustrated in FIGS. 16(a) and 16(b), by changing some ofthe photochromic mirrors 403 to become a mirror state, distances fromthe LCD 30 b to the lens 42 a are differentiated for every area, wherebyit is possible to change the display distance of the virtual image inaccordance with this.

FIG. 17 is a view illustrating an outline of a configuration example ofthe photochromic mirrors 403. Each of the photochromic mirrors 403 is amember, such as a film, a sheet, or a glass, which can be controlledbetween a transparent state and a mirror state by means of electricity.The photochromic mirrors 403 each of which becomes the transparent statetransmit video from the LCD 30 b, but only the photochromic mirrors 403each of which becomes the mirror state reflect the video toward adirection to the lens 42 a. In the present embodiment, the photochromicmirror ON/OFF controller 283 controls the plurality of photochromicmirrors 403 arranged in the matrix shape when viewed from thecross-sectional direction so that only one photochromic mirror 403 ineach row and each column (that is, each area) becomes the mirror state.

Specifically, for example, as illustrated in FIG. 16(a), with respect tothe area corresponding to the nearest column of the photochromic mirrors403 from the lens 42 a, by setting only the photochromic mirror 403 inthe lowermost row to the mirror state and setting the other photochromicmirrors 403 to the transparent state, it is possible to minimize anoptical path length from the LCD 30 b to the lens 42 a. This makes itpossible to set the display distance of the corresponding virtual imageto the nearest position. Conversely, with respect to the areacorresponding to the farthest column of the photochromic mirrors 403from the lens 42 a, by setting only the photochromic mirror 403 in theuppermost row to the mirror state and setting the other photochromicmirrors 403 to the transparent state, it is possible to maximize theoptical path length from the LCD 30 b to the lens 42 a. This makes itpossible to set the display distance of the corresponding virtual imageto the farthest distance.

Further, for example, as illustrated in FIG. 16(b), with respect to thearea corresponding to the nearest column of the photochromic mirrors 403from the lens 42 a, only the photochromic mirror 403 in the uppermostrow is set to the mirror state. Further, with respect to the areacorresponding to the second nearest column of the photochromic mirrors403 from the lens 42 a, only the photochromic mirror 403 in the lowestrow is set to the mirror state, and the other photochromic mirrors 403are set to the transparent state. This makes it possible to relativelyshorten the optical path lengths of these areas from the LCD 30 b to thelens 42 a, and to set the display distances of the corresponding virtualimage to the nearby positions. Conversely, with respect to the areacorresponding to the farthest column of the photochromic mirrors 403from the lens 42 a, by setting only the photochromic mirror 403 in themiddle row to the mirror state and setting the other photochromicmirrors 403 to the transparent state, it is possible to relativelylengthen the optical path length from the LCD 30 b to the lens 42 acompared with the other areas. This makes it possible to set the displaydistance of the corresponding virtual image to the farthest distance.

Note that in the examples of FIG. 16 and FIG. 17, the number ofphotochromic mirrors 403 is also not limited to the 3×3 matrix asillustrated in FIGS. 16 and 17, and can be set appropriately inaccordance with the number of areas.

[Movable Diffuser Plate]

FIG. 18 is a view illustrating an outline of an example of theadjustment of the display distance by using a movable diffuser plate inthe head up display according to the present embodiment. In the exampleof FIG. 18, after video projected from the projector 30 a is formed on amovable diffuser plate (movable diffuser) 41 c, the video is madeincident to the mirror 52 via the lens 42 a.

Here, the movable diffuser plate 41 c can move and/or rotate along anoptical axis direction by means of the diffuser plate moving unit 284and the diffuser plate moving mechanism 404 illustrated in FIG. 5. Adisplay position of a virtual image based on video projected from theprojector 30 a is determined in accordance with a distance and/or a leanbetween the movable diffuser plate 41 c and the lens 42 a. Therefore, bymoving and/or rotating the movable diffuser plate 41 c to change a focaldistance, it is possible to change the display distance of the virtualimage.

Specifically, by moving the movable diffuser plate 41 c to a positionnear the lens 42 a and/or rotating the movable diffuser plate 41 c so asto approach the lens 42 a, it is possible to set the display distance ofthe virtual image to a nearby position. Conversely, by moving themovable diffuser plate 41 c to a position far from the lens 42 a and/orrotating the movable diffuser plate 41 c so as to go away from the lens42 a, it is possible to set the display distance of the virtual image toa faraway position.

[Movable Optical Filter]

FIG. 19 is a view illustrating an outline of an example of theadjustment of the display distance by using a movable optical filter inthe head up display according to the present embodiment. In the exampleof FIG. 19, a movable optical filter 43 a is mounted between the lens 42a and the diffuser plate (diffuser) 41 b. As illustrated in FIGS. 19(a)and 19 (b), by inserting or removing the movable optical filter 43 awith respect to an optical path, focal distances are changed for everyarea to change a display distance of a virtual image.

The optical filter is a member having a property to change a focaldistance by means of a single body or combination of opticalcomponent(s) such as a lens. In the present embodiment, by combining aplurality of optical filters each having a different refractive indexfrom that of the others, one optical filter with areas each having adifferent refractive index is formed to be constituted as the movableoptical filter 43 a that is allowed to be inserted and removed withrespect to the optical path. Since focal distances of the optical filterare different for each area, the optical filter moving unit 285 and theoptical filter moving mechanism 405 illustrated in FIG. 5 inserts andremoves the movable optical filter 43 a with respect to the opticalpath, whereby it is possible to change the display distance of thevirtual image for each area.

Specifically, for example, as illustrated in FIG. 19(a), by insertingthe whole movable optical filter 43 a into the optical path, it ispossible to set the focal distance of the optical filter 43 acorresponding to an area of the lowest stage to the shortest distance,and set the display distance of the virtual image to the farthestdistance. Further, by setting the focal distance of the optical filter43 a corresponding to an area of the uppermost stage, it is possible toset the display distance of the virtual image to the nearby position.Further, for example, as illustrated in FIG. 19(b), by removing a partof the movable optical filter 43 a so that video corresponding to thearea of the lowest stage does not pass through the optical filter 43 a,the display distance of the virtual image for this area is determined inaccordance with a distance between the diffuser plate 41 b and the lens42 a, whereby it is possible to set the display distance of the virtualimage to a faraway position than that of the other areas for which videopasses through the optical filter 43 a.

Note that in the example of FIG. 19, the number of areas whose focaldistance are different from each other in the movable optical filter 43a is also not limited to three as illustrated in FIG. 19, and can be setappropriately in accordance with the number of areas.

[Comb-Shaped Optical Filter]

FIG. 20 is a view illustrating an outline of an example of theadjustment of the display distance by using a comb-shaped optical filterin the head up display according to the present embodiment. In theexample of FIG. 20, after video projected from the projector 30 a isformed on the diffuser plate (diffuser) 41 b, the video is made incidentto the mirror 52 via a comb-shaped optical filter 43 b and the lens 42a.

The comb-shaped optical filter 43 b has a similar function to that of alens. The comb-shaped optical filter 43 b is a member in which anoptical filter portion is provided in a comb-shaped manner, and that canchange a display distance of a virtual image in accordance with a focaldistance. As illustrated in FIG. 20, for example, by handling theoptical filter portion and a portion that is not the optical filter by aline unit (which is not limited to every one line, and can beconstituted by every any number of lines) of video projected from theprojector 30 a, it is possible to change the display distance of thevirtual image for each line unit.

Specifically, it is possible to set the display distance of the virtualimage based on the video of the line corresponding to the optical filterportion to a nearby position, and to set the display distance of thevirtual image based on the video of the line corresponding to theportion that is not the optical filter to a faraway position.

As explained above, according to the head up display of the firstembodiment of the present invention, even in a case where the virtualimage cannot be superimposed onto the actual view in front of thevehicle 2 depending upon the traveling status of the vehicle 2, itbecomes possible to realize the AR function to appropriately superimposethe virtual image onto the actual view ahead by dynamically adjustingthe display position of the display area for the virtual image itself inthe vertical direction. Moreover, it also becomes possible toappropriately adjust the display distance of the virtual image inaccordance with the traveling status and the like.

Second Embodiment

<Problems>

In the head up display according to the first embodiment describedabove, as illustrated in the example of FIG. 9, in a case where there isa difference between a gradient of a current traveling position and agradient of a road ahead, a position of a display area for a virtualimage in a vertical direction is adjusted in a manner of hardware by themirror adjusting process to control an angle of a mirror 52 by a mirrordriver 50 in accordance with a traveling status of a vehicle 2. Further,in a case where a vibration amount of the vehicle 2 exceeds apredetermined threshold value, a vibration correcting process to offseta display position of video in a display area up and down in a manner ofsoftware is executed. These processes allow the virtual image to beappropriately superimposed onto an actual view ahead against a lean orvibration of the vehicle 2.

On the other hand, there are various modes of the lean or vibration(hereinafter, also collectively referred to as “jolting”) of the vehicle2, and these may often occur so as to be combined. Therefore, there isneed to flexibly handle various kinds of modes of jolting of the vehicle2 so that display of a virtual image can be adjusted. FIG. 21 to FIG. 23are views each illustrating an outline of an example of jolting of thevehicle 2 and a normal display state of the virtual image.

In FIG. 21, as well as the example of FIG. 9, in each of left and rightviews, an upper stage schematically illustrates a state of a status of aroad on which the vehicle 2 is traveling and a status of a viewing fieldof the driver 5 when viewed from a side surface thereof. Further, alower stage schematically illustrates, in each of the states, a statusof an actual view in front of the vehicle exterior, which is viewed bythe driver 5, and a position of a display area 6 (a rectangle indicatedby a dashed frame) of the virtual image displayed so as to besuperimposed onto the actual view.

A view at a right side of FIG. 21 illustrates a case where the vehicle 2is traveling on a substantially flat road as illustrated in the view ofthe upper stage. In this case, as illustrated in the view of the lowerstage, in order to superimpose the virtual image (in the example of FIG.21, a mark or an image of an exclamation point) onto the actual view infront of the vehicle exterior (in the example of FIG. 21, a vehicleahead that travels on the road) by the AR function and display it, theposition of the display area 6 for the virtual image in a verticaldirection may be a normal position. Namely, the position of the displayarea 6 illustrated in the view of the lower stage at the right sidebecomes a basic display position of the display area 6 in the verticaldirection.

On the other hand, a view at a left side illustrates a case where thereis unevenness on a road surface at a current location of the vehicle 2.In this case, in a case where any adjustment or the like is not executedparticularly, as illustrated in the view of the lower stage, positionsof the virtual image and the display area 6 therefor are also shaken inthe relatively vertical direction with the actual view in front of thevehicle exterior in accordance with jolting of the vehicle 2 in thevertical direction (a pitch direction). Herewith, the virtual image isdisplayed at a position at which the virtual image is not superimposedonto the actual view in front of the vehicle exterior, whereby anuncomfortable feeling given to the driver 5 becomes stronger. Further,since the virtual image is shaken, the driver 5 who views this maybecome carsick.

Further, in FIG. 22, in each of left, central, and right views, an upperstage schematically illustrates a state that a status of an inclinationof a road on which the vehicle 2 is traveling when viewed from the rearof the vehicle 2. Further, a lower stage schematically illustrates astatus of an actual view in front of vehicle exterior viewed by thedriver 5 and a position of a display area 6 for the virtual imagesuperimposed and displayed on the actual view in each state. Asillustrated in the view of the upper stage, the central view of FIG. 22illustrates a case where the vehicle 2 is traveling on a substantiallyflat road. In this case, as illustrated in the view of the lower stage,the display area 6 of the virtual image is displayed at a basic displayposition without being inclined.

On the other hand, as illustrated in the view of the upper stage, theright and left views respectively illustrate cases where road surfacesat a current location of the vehicle 2 are inclined in a horizontaldirection. In this case, in a case where any adjustment or the like isnot executed particularly, as illustrated in the view of the lowerstage, the virtual image and the display area 6 are also inclined in ahorizontal direction in accordance with an inclination of the vehicle 2in the horizontal direction (a roll direction). Herewith, as well as theexample of FIG. 21, the virtual image is displayed at a position atwhich the virtual image is not superimposed onto the actual view infront of the vehicle exterior, whereby an uncomfortable feeling given tothe driver 5 becomes stronger. Further, the driver 5 may become carsickdue to shaking of the virtual image.

Moreover, FIG. 23 illustrates a case where jolting of the vehicle 2 in avertical direction (a pitch direction) and an inclination in ahorizontal direction (a roll direction) occur at the same time so as tobe combined. In this case, as illustrated in the view of the lowerstage, the virtual image and the display area 6 are inclined in thehorizontal direction, and the display position is also shaken in thevertical direction.

<Outline of Control>

In order to handle these statuses flexibly and cause the virtual imageto be appropriately superimposed onto the actual view ahead, the head updisplay according to the second embodiment of the present invention isimproved so that the mirror adjusting process and the vibrationcorrecting process (Steps S24 and S25 in FIG. 7) in the head up displayaccording to the first embodiment correspond to various kinds of modesof jolting of the vehicle 2 more flexibly.

Specifically, in the present embodiment, in the hardware configurationfor acquiring the vehicle information 4, which has been described abovewith reference to FIG. 3, two posture sensors 111 are provided toseparately detect jolting in the vertical direction (the pitchdirection) and an inclination in the horizontal direction (the rolldirection). Namely, a first posture sensor 111 (not illustrated in thedrawings) detects jolting in the vertical direction (the pitchdirection) (which is grasped by a lean of the vehicle 2 in a front-backdirection), and a second posture sensor 111 detects an inclination ofthe vehicle 2 in the horizontal direction (the roll direction). Thedisplay position of the virtual image in the vertical direction is thenadjusted in accordance with an angle detected by the first posturesensor 111. Further, a lean of display of the virtual image in thehorizontal direction is adjusted in accordance with an angle detected bythe second posture sensor 111. Moreover, by combining these kinds ofadjustment, it is possible to execute the adjustment in the verticaldirection and the adjustment of the lean in the horizontal direction atthe same time.

A mounting position of the first and second posture sensors 111 on thevehicle 2 is not limited particularly. However, both posture sensors 111can be mounted in a housing of the AR-HUD 1, for example. Alternatively,both posture sensors 111 may be mounted on a predetermined place outsidethe housing, and detected information may be taken in the AR-HUD 1 bythe vehicle information acquiring unit 10. In this case, both posturesensors 111 may be mounted on a driver's seat or the like so that valuesacquired by approximating shaking and inclination that occur on thedriver 5 can be acquired. The first and second posture sensors 111 maybe mounted on both the inside of the housing of the AR-HUD 1 and theoutside of the housing of the driver's seat, the ECU 21 may analyzejolting, a lean, posture and the like of the vehicle 2 on the basis ofinformation acquired from these, and execute the adjustment on the basisof this. Further, three or more posture sensors may be mounted to detectthe jolting of the vehicle 2 in more detail.

In order to detect the jolting of the vehicle 2 in the verticaldirection (the pitch direction) and the inclination in the horizontaldirection (the roll direction), alternatively to use the two first andsecond posture sensors 111 as the hardware configuration to acquire thevehicle information 4, or in addition to the first and second posturesensors 111, the camera (vehicle exterior) 116 can be used. Namely, bycalculating a shift amount at predetermined time intervals in image dataof the actual view ahead photographed by the camera (vehicle exterior)116, it is possible to detect jolting of the vehicle 2. The otherconfiguration and the processing content of the head up displayaccording to the present embodiment are basically similar to thoseillustrated in the first embodiment described above. Thus, repeatedexplanation will be omitted.

FIG. 24 is a view illustrating an outline of an example in which thedisplay position of the virtual image is adjusted up and down in thehead up display according to the present embodiment. FIG. 24 (a)illustrates, in the similar example to the example of FIG. 21 describedabove, for example, an example of a case where the vehicle 2 jolts in anupward direction (that is, in a pitch direction in which the front partof vehicle 2 is relatively raised up compared with the rear part of thevehicle 2) while the vehicle 2 is traveling on a bumpy road in the viewof a left side. In the present embodiment, in this case, adjustment ofthe display position of the display area 6 itself for the virtual imageis not executed temporarily. Therefore, as well as the example of FIG.21, the display area 6 itself is relatively moved in the upwarddirection with respect to the actual view in front of the vehicleexterior with jolting of the vehicle 2 in the upward direction (in thepitch direction in which the front part of vehicle 2 is relativelyraised up compared with the rear part of the vehicle 2).

At this time, in a case where the adjustment is not executedparticularly as the example illustrated in FIG. 21, the virtual imagedisplayed in the display area 6 (in the example of FIG. 21, a mark or animage of an exclamation point) is also relatively moved in the upwarddirection along with the display area 6. On the other hand, in thepresent embodiment illustrated in the example of FIG. 24(a), asillustrated in the view of the lower stage, it is controlled so that anabsolute display position of the virtual image, which is displayedtherein, is not changed from an original display position as much aspossible even though the display area 6 for the virtual image is movedin the upward direction. Namely, when the ECU 21 generates video datarelated to the virtual image, the ECU 21 controls, in a manner ofsoftware, so that a relative display position of the virtual image ismoved downward in the display area 6 that is moved in the upwarddirection.

FIG. 24(b) illustrates, in an example similar to that of FIG. 24(a), acase where since jolting of the vehicle 2 in the upward direction (inthe pitch direction in which the front part of vehicle 2 is relativelyraised up compared with the rear part of the vehicle 2) is large, thevirtual image protrudes (or goes out) from a lower end (or end portion)of the display area 6 by moving the display position of the virtualimage to an appropriate position, at which the virtual image can besuperimpose onto the actual view ahead, in a downward direction. In thepresent embodiment, in this case, when the ECU 21 generates the videodata related to the virtual image, as illustrated in the view of a leftside of FIG. 24(b), it is controlled so that movement of the virtualimage in the downward direction is stopped at the lower end of thedisplay area 6 because the virtual image does not protrude from thelower end of the display area 6 (it is desirable that at least a half ormore of the virtual image is displayed in the display area 6). Thismakes it possible to avoid a situation that a half of the virtual imagecannot be viewed or a situation that the virtual image cannot be viewedat all. This allows the driver 5 to always recognize the virtual imagevisually, whereby anxiety or an uncomfortable feeling generated on thedriver 5 is reduced.

Note that FIG. 24 illustrates, as an example, the case where the vehicle2 jolts in the upward direction (in the pitch direction in which thefront part of vehicle 2 is relatively raised up compared with the rearpart of the vehicle 2). However, in a case where the vehicle 2 jolts ina downward direction (that is, in a pitch direction in which the frontpart of the vehicle 2 is relatively dropped down compared with the rearpart of the vehicle 2), a control that is contrary to the above is to beexecuted. Further, with respect to an inclination of the vehicle 2 in ahorizontal direction (a roll direction), it is possible to adjust thedisplay position of the virtual image by rotating display of the virtualimage in the horizontal direction in the similar way.

In the example illustrated in FIG. 24, the ECU 21 does not executeadjustment of the display position of the display area 6 itself for thevirtual image and the inclination (hereinafter, also collectivelyreferred to as “display state”) with respect to jolting of the vehicle2, but adjusts only the display state of the virtual image displayedtherein in the manner of software. Therefore, even in a case wherevibration and an inclination are combined on the vehicle 2complicatedly, it is possible to flexibly adjust the display state ofthe virtual image. On the other hand, as illustrated in the firstembodiment, by rotating the mirror 52 by means of the mirror driver 50,it is also possible to move the display area 6 itself for the virtualimage in the vertical direction. This makes it possible to control thedisplay position in the manner of hardware so that the display positionof the display area 6 is not shaken even in a case where jolting of thevehicle 2 occurs.

FIG. 25 is a view illustrating an outline of an example in which thedisplay position of the virtual image is adjusted up and down in thehead up display according to the present embodiment. FIG. 25(a)illustrates, in an example similar to the example of FIG. 21 describedabove, that even in a case where the vehicle 2 jolts in a verticaldirection (a pitch direction) while the vehicle 2 is traveling on abumpy road of the view of a left side, in the present embodiment, thedisplay area 6 is adjusted so that a position of the display area 6itself for the virtual image is not shaken in the vertical direction.

FIG. 25(b) illustrates an example in which in order to execute theadjustment as illustrated in FIG. 25(a), by rotating a mirror 52 arounda rotation axis of a horizontal direction, a direction to reflect videoprojected from the video display 30 is changed to move a position to beprojected onto a windshield 3 in the vertical direction. The method ofadjusting the position of the display area 6 itself for the virtualimage in the vertical direction is not limited to the method of rotatingthe mirror 52. As illustrated in FIG. 25(c), even by moving the positionof the mirror 52 in an optical axis direction forward or backward, it isalso possible to move a position at which the video projected from thevideo display 30 is projected onto the windshield 3 in the verticaldirection. The rotation of the mirror 52 illustrated in FIG. 25(b) andthe movement of the mirror 52 illustrated in FIG. 25(c) in thefront-back direction may be combined.

FIG. 26 is a view illustrating an outline of a configuration example ofthe mirror 52 and the mirror driver 50 in the head up display accordingto the present embodiment. FIG. 26(a) is a perspective view illustratinga concrete configuration example of the mirror 52 and the mirror driver50. The mirror driver 50 is composed of a supporting member 54 a and amirror holding member 54 b, for example. These members and the mirror 52are combined via mirror driving shafts A (53 a) to C (53 c) asillustrated in FIG. 26. A position and an angle of the mirror 52 can beadjusted by rotating each of the mirror driving shafts A (53 a) to C (53c).

FIG. 26 (b) is a view schematically illustrating a state where themirror 52 and the mirror driver 50 are fixed in the housing of theAR-HUD 1 when viewed from a side surface thereof. For convenience ofexplanation, x, y, and z axes are also displayed together. For example,by rotating the mirror driving shafts A (53 a) and C (53 c)counterclockwise by using the y axis as an axis and rotating the mirrordriving shaft B (53 b) clockwise from a state illustrated in a centralview, as illustrated in a view of a left side, it is possible to movethe position of the mirror 52 in a minus z direction while maintainingthe angle of the mirror 52.

Conversely, by rotating the mirror driving shafts A (53 a) and C (53 c)clockwise by using the y axis as an axis and rotating the mirror driverB (53 b) counterclockwise, as illustrated in a view of a right side, itis possible to move the position of the mirror 52 in a plus z directionwhile maintaining the angle of the mirror 52. Further, separately fromthese kinds of movement, or in addition to these kinds of movement, byadjusting rotation of the mirror driving shaft C (53 c) and the like, itis possible to adjust the angle of the mirror 52.

Further, a rotary drive mechanism (not illustrated in the drawings)rotates the whole mirror driver 50 or the whole AR-HUD 1 by using the zaxis as an axis, whereby it is possible to rotate the display area 6itself for the virtual image that is reflected by the mirror 52 andprojected onto the windshield 3 in a horizontal direction. FIG. 27 is aview illustrating an outline of an example in which right and left leanof the display area for the virtual image is adjusted in the head updisplay according to the present embodiment. FIG. 27(a) illustrates thateven in a case where the vehicle 2 is inclined in the horizontaldirection (a roll direction) while the vehicle 2 is traveling on aninclined road, in the present embodiment, a lean of the display area 6itself for the virtual image in the horizontal direction is adjusted soas not to be shaken. At this time, as illustrated in FIG. 27(b), forexample, the whole AR-HUD 1 is rotated in an opposite direction by usingthe z axis as an axis so as to be inclined in accordance with aninclination of the vehicle 2. This makes it possible to rotate thedisplay area 6 itself for the virtual image in the horizontal directionand adjust a display position thereof, whereby it is possible to controlthe display area 6 in the manner of hardware so that a shift isgenerated in the lean of the display area 6 with respect to the actualview ahead. It is also possible to move the whole AR-HUD 1 in thefront-back direction (a z axis direction). In the example describedabove, in a case where the whole AR-HUD 1 is not rotated or moved andthe mirror driver 50 mounted therein is rotated or moved, there is needto provide a rotary drive mechanism or a moving mechanism inside theAR-HUD 1. Therefore, a demerit that a body size of the AR-HUD 1 becomeslarger may be generated. There is only a limited space in a dashboard ofthe vehicle 2. Thus, a case where it is difficult to store the AR-HUD 1in the dashboard may occur when the body size thereof becomes larger.

On the other hand, in a case where the AR-HUD 1 itself is rotated, thereis need to provide a rotating mechanism outside the body of the AR-HUD1. However, in this case, the size of the AR-HUD 1 itself does notbecome larger. Further, even though the rotating mechanism providedoutside the AR-HUD 1 is considered to be included, it is possible toimplement a mechanism for adjusting the lean of the display area 6 withspace saved compared with the case where the mirror driver 50 is rotatedor moved inside the AR-HUD 1.

It is possible to combine and use the method of adjusting the displaystate of the virtual image in the manner of software by the ECU 21 andthe method of adjusting the display state of the display area 6 itselffor the virtual image in the manner of hardware by rotating or movingthe mirror 52. FIG. 28 is a view illustrating an outline of an examplein which the virtual image and a display position of the display area 6are adjusted up and down in the head up display according to the presentembodiment. Here, as well as the example of FIG. 24(b) described above,even though jolting of the vehicle 2 is large and the virtual imageprotrudes from the display area 6 only by the adjustment of the displayposition of the virtual image in the manner of software, the displayposition of the display area 6 itself is adjusted up and down byrotating or moving the mirror 52. By adjusting the display position ofthe virtual image within the display area 6, whose display position isadjusted in this manner, in the manner of software, it is possible toappropriately superimpose the virtual image onto the actual view aheadso that the virtual image falls in the display area 6 and display thevirtual image.

In the example of FIG. 28, with respect to the virtual image (in theexample of FIG. 28, a mark or an image of an exclamation point)superimposed onto the actual view of the vehicle ahead and displayed inaccordance with jolting of the vehicle 2 in the vertical direction (thepitch direction), the display position thereof is adjusted, andsuperimposing onto the actual view of the vehicle ahead is continued.However, control content related to the adjustment of the displayposition may be changed in accordance with a property of the virtualimage. For example, a virtual image such as meters and gauges, which isnot required to be superimposed onto the actual view ahead, (forexample, display of speed of “35 km” in the example of FIG. 28) may makean exception for the adjustment of the display position in the displayarea 6 due to jolting of the vehicle 2 (that is, display of such avirtual image is shaken with jolting of the vehicle 2).

Further, the similar control to the virtual image for which there isneed to be superimposed onto the actual view ahead may be executed forsuch a virtual image in accordance with the property of the virtualimage, and display thereof may be adjusted in the display area 6. Forexample, in order to monitor a blind spot at the rear and the side ofthe vehicle, video of the spot may be acquired by the camera (vehicleexterior) 116 as a so-called electron mirror, and this may be displayedin the display area 6 as a virtual image. In this case, the virtualimage of the monitored video is not displayed in the entire display area6, but is generally displayed in a partial area of any of four cornersin the form of a slave screen, for example. Therefore, since the displayitself is relatively small and the display of the virtual image is alsoshaken with jolting of the vehicle 2, the driver 5 further hardlyrecognizes the display content visually.

The virtual image related to the monitored video of such an electronmirror is not necessarily superimposed onto the actual view ahead.However, the similar control may be executed for the virtual image thatneeds to be superimposed to adjust the virtual image by moving displayof the virtual image or the display area 6 in the vertical direction orrotating it in the horizontal direction. Note that, in this case,priority of adjustment may be lower for the virtual image that needs tobe superimposed onto the actual view ahead.

Further, in the present embodiment, the display state of the virtualimage is first adjusted in the display area 6 in the manner of software.In a case where the virtual image cannot be displayed in the displayarea 6 appropriately because a quantity of jolting is large, it iscontrolled so as to adjust the display state of the display area 6itself for the virtual image in the manner of hardware by rotating ormoving the mirror 52. However, a reverse control thereof can beexecuted. Namely, the ECU 21 first adjusts the display area 6 for thevirtual image in the manner of hardware by rotating or moving the mirror52 so that shaking does not occur in the display area 6 itself. Then, ina case where the mirror 52 is adjusted to the maximum but the adjustmentis insufficient due to limitation of a movable range that is generatedfrom restriction on structures of the mirror 52 and the mirror driver50, the ECU 21 may further adjust the display state of the virtual imagein the manner of software to complement the insufficiency.

Further, in the above example of the present embodiment, in a case wherejolting of the vehicle 2 is detected, the shaking of the displayposition of the virtual image with respect to the actual view ahead isadjusted. However, a little shaking may be admitted. The ECU 21 maycontrol to execute adjustment of the display state (by restricting theshaking so as not to become the threshold value or more) only in a caseof shaking of a predetermined threshold value or more. This is because,depending upon the driver 5, there is a case where the driver 5 may feelnormal to experience shaking of the virtual image with which jolting ofthe vehicle 2 corresponds under a little jolting of the vehicle 2.

<Improvement of Real-Time Property>

By adjusting the virtual image and the display state of the display area6 in the virtual image by means of the configuration as described above,it is possible to control the virtual image so that jolting does notoccur in the display of the virtual image. Here, jolting of the vehicle2 due to unevenness on a road surface or the like may become a shortcycle. In this case, for example, in a case where intended jolting hasalready been stopped or jolting is changed in a different direction atthe time of completion of adjustment of the display state of the virtualimage by the method as described above, the adjustment becomesirrelevant. Therefore, a high real-time property is required toimmediately handle detected jolting as much as possible.

Thus, in the present embodiment, the ECU 21 does not calculate a movingamount or a rotating amount (a lean) of the display of the virtual imageon the basis of values of angles respectively detected by the first andsecond posture sensors 111 (that is, values indicating jolting of thevehicle 2) each time, but refers to a table to directly determine themoving amount or the rotating amount based on the detected angle,whereby the processing can be simplified. In the present embodiment, acorrespondence relationship between various angles in a range in which agiven angle can be taken and moving amounts and/or rotating amounts ofdisplay of the virtual image is set in advance as a virtual imageadjusting amount table, and the virtual image adjusting amount table isheld in the nonvolatile memory 23 or the like.

FIG. 29 is a view illustrating an outline of an example of the virtualimage adjusting amount table in the head up display according to thepresent embodiment. Here, FIG. 29 shows a state that a moving amount anda rotating amount of display of the virtual image with respect todetected angles of the first and second posture sensors 111 arerespectively set. By referring to this table, it is possible to directlyacquire the moving amount or the rotating amount, which corresponds tothe angles respectively detected by the first and second posture sensors111, of the display for the virtual image.

Note that specification and/or a property of the display area 6 in theAR-HUD 1 are different depending upon a type of the AR-HUD 1 and are notdefined uniquely. Therefore, the content of this table may be differentin accordance with specification or the like of the AR-HUD 1. Forexample, in the example of the table for the first posture sensor 111 ata left side illustrated in FIG. 29, a moving amount is rounded to ±6when the angle exceeds ±10. This is because the virtual image protrudesfrom the display area 6 by moving the virtual image to ±6 or more (andless) in the AR-HUD 1. Therefore, the moving amount is limited to thevalue or less. Further, the moving amount is set to zero when the angleis smaller than ±5. This indicates that the angle smaller than ±5 isminute and is not detected as shaking (that is, a threshold value todetect jolting of the vehicle 2 is ±5).

The content of the virtual image adjusting amount table may be differentdepending upon a property of the vehicle 2 on which the AR-HUD 1 ismounted in addition to specification and the like of each AR-HUD 1. Forexample, the optimum moving amount or the optimum rotating amount may bedifferent depending upon a length or a width of a vehicle body,specification of a suspension, and the like. Therefore, it is desirablethat the setting content of the virtual image adjusting amount table hasthe optimum values that are defined on the basis of the specification ofthe AR-HUD 1 itself, the specification of the vehicle 2 on which theAR-HUD 1 is mounted, and the like. For example, plural patterns ofvirtual image adjusting amount tables, in each of which an appropriatedefault value is set in advance for each typical specification of thevehicle 2, are held for each AR-HUD 1, and one virtual image adjustingamount table to be used may be selected or determined automatically ormanually in accordance with specification of the vehicle 2 on which theAR-HUD 1 is actually mounted.

Shift of the display state of the virtual image superimposed onto theactual view ahead and displayed, and an uncomfortable feeling that thedriver 5 feels such as the degree of shaking of display may be differentdepending upon taste of the driver 5. Further, even in a case where theAR-HUD 1 with the same specification is mount on the same vehicle 2, thedegree that the driver 5 desires to adjust display of the virtual imagewith respect to jolting or a lean may be different depending upon aproperty of a road surface on which the vehicle 2 is often traveling ona daily basis such as traveling to work (whether a ratio of a flat roadis large or not, or whether a ratio of a bumpy road is large or not).Further, for example, the degree of shaking or shift of display maychange due to circumstances such as a situation that the driver 5changes a suspension or wheels of the vehicle 2.

Therefore, in the present embodiment, a method in which the driver 5 canset the degree of adjustment of the display state of the virtual imageand sensitivity in accordance with taste of the driver 5 is provided.FIG. 30 is a view illustrating an outline of an example in which adegree or sensitivity of adjustment of the display state of the virtualimage is allowed to be set in the head up display according to thepresent embodiment. FIG. 30(a) illustrates an example of a menu screenthat the AR-HUD 1 displays in the display area 6 in order to receivesettings by the driver 5.

Here, for example, it is possible to receive settings to increase ordecrease a moving amount when display of the virtual image is moved asan “adjustment amount of display position”, and settings to increase ordecrease a threshold value of an amount of shaking to execute adjustmentof the display state of the virtual image as an “adjustment sensitivityof display position”. In the “adjustment amount of display position”,for example, by setting “strengthen adjustment” as illustrated in FIG.30(a), it is possible to set the virtual image to be further moved orrotated in a case where the driver 5 feels lack of an adjusting amountof the display state by the AR-HUD 1 with respect to shaking of displayof the virtual image against the actual view ahead with jolting of thevehicle 2.

At this time, for example, in the setting content of the virtual imageadjusting amount table as illustrated in FIG. 30(b), the ECU 21automatically makes the absolute value of the moving amount of displayof the virtual image with respect to the angle detected by the firstposture sensor 111 larger by a predetermined value from the originalcontent. A set value of the rotating amount of the virtual image withrespect to the angle detected by the second posture sensor 111 is alsosimilar. Note that the table whose setting content is changed is held inthe nonvolatile memory 23, but the original setting content beforechange is also held in the nonvolatile memory 23 at that time. Thismakes it possible to “restore” the setting content as illustrated in ascreen example of FIG. 30(a).

Similarly, in the “adjustment sensitivity of display position” of themenu screen illustrated in FIG. 30(a), for example, by setting “weakensensitivity” as illustrated in FIG. 30(a), in a case where the driver 5feels that the display state of the virtual image is not to be adjustedwhen jolting of the vehicle 2 is little and the virtual image may shapenaturally, the threshold value to determine as jolting can be set to belarger, whereby it is possible to set the virtual image so that thedisplay state of the virtual image is not adjusted under little joltingof the vehicle 2. Specifically, in the virtual image adjusting amounttable, the angle at which the moving amount or the rotating amount ofdisplay of the virtual image is not zero is set to be larger (that is, arange of the angle in which the moving amount or the rotating amount iszero is set to be larger).

Note that as illustrated in the example of the menu screen of FIG.30(a), the method of setting the degree of adjustment or sensitivity bythe driver 5 is not limited to simple setting by specifying “strengthen”or “weaken”. For example, a detailed setting mode in which the driver 5can directly set or edit the values of the moving amount and therotating amount for each angle set in the virtual image adjusting amounttable may be provided. Further, results of jolting and/or a lean of thevehicle 2 actually detected by the first and second posture sensors 111may be accumulated, and the ECU 21 may analyze this to automatically setor recommend the optimum set value.

<Processing Content>

FIG. 31 is a flowchart illustrating an outline of an example of a normaloperation executed by the head up display according to the presentembodiment. As described above, in the present embodiment, the mirroradjusting process (S24) and the vibration correcting process (S25) inthe processing flow for the normal operation illustrated in FIG. 7according to the first embodiment are improved to become a display stateadjusting process (S34). Here, jolting and/or a lean of the vehicle 2due to unevenness and the like of a road surface on which the vehicle 2is currently traveling is suppressed by moving and/or rotating thevirtual image or the display state of the display area 6 to adjust thevirtual image so that the virtual image is appropriately superimposedonto the actual view ahead of the vehicle 2.

On the other hand, the mirror adjusting process (S24) in the processingflow illustrated in FIG. 7 according to the first embodiment is toadjust the display position of the display area 6 with respect to agradient of the road surface on which the vehicle 2 is currentlytraveling by taking into consideration a gradient of the road ahead.This may be compatible with the method of adjusting the display state ofthe virtual image according to the present embodiment. Therefore, thedisplay state adjusting process (S34) may be made by improving thevibration correcting process (S25) while maintaining the mirroradjusting process (S24) in the processing flow illustrated in FIG. 7.Note that the other steps in the processing flow illustrated in FIG. 31are similar to the corresponding steps in the processing flowillustrated in FIG. 7 according to the first embodiment. Therefore,repeated explanation will be omitted.

FIG. 32 is a flowchart illustrating an outline of an example of adisplay state adjusting process executed by the head up displayaccording to the present embodiment. When the display state adjustingprocess is started, the ECU 21 first acquires information on anglesdetected by the first and second posture sensors 111 on the basis of thevehicle information 4 (S341). The ECU 21 then determines whether a valueof each of the angles is caused by jolting of the vehicle 2 (that is,vibration and/or a lean with a short cycle) or not (S342). For example,in a case where a difference between an angle detected at the time ofprevious processing (or before a fixed time) and an angle detected thistime is equal to or more than a predetermined threshold value, it isdetermined that the value is caused by jolting of the vehicle 2. Namely,in a case where the vehicle 2 itself is inclined for a fixed amount inthe front-back direction (the pitch direction) or the horizontaldirection (the roll direction) but such a state is continued, it isdetermined that a state of the vehicle 2 does not change and the vehicle2 does not jolt.

In a case where any of the angles detected by the first and secondposture sensors 111 is not caused by jolting of the vehicle 2 (S342:No), the ECU 21 terminates the display state adjusting process as it is.On the other hand, in a case where at least one is caused by jolting ofthe vehicle 2 (S342: Yes), the ECU 21 subsequently starts a loopprocessing to execute a process for each of the first and second posturesensors 111 that detects the angle caused by jolting of the vehicle 2(S343). In the loop processing, the ECU 21 first refers to the virtualimage adjusting amount table held in the nonvolatile memory 23, andacquires a value of the adjusting amount (a moving amount in thevertical direction or a rotating amount in the horizontal direction) ofthe display state of the virtual image corresponding to the angledetected by the target posture sensor (S344).

The ECU 21 then determines whether the adjusting amount of the displaystate of the virtual image exceed a predetermined threshold value or not(S345). In the present embodiment, as described above, the ECU 21 firstcontrols to adjust the display state of the virtual image in the displayarea 6 in the manner of software. In a case where an amount of joltingis large and the virtual image cannot be displayed in the display area 6appropriately, the ECU 21 controls to adjust the display state of thedisplay area 6 itself for the virtual image in the manner of hardware byrotating and/or moving the mirror 52. Therefore, the ECU 21 hereindetermines whether the adjusting amount of the virtual image is as largeas the virtual image cannot be displayed in the display area 6appropriately or not. In a case where the adjusting amount of thedisplay state of the virtual image is larger than the predeterminedthreshold value (S345: Yes), the ECU 21 executes a mirror stateadjusting process for adjustment by rotating and/or moving the mirror 52(S346).

FIG. 33 is a flowchart illustrating an outline of an example of themirror state adjusting process executed by head up display the accordingto the present embodiment. The processing content herein is basicallysimilar to the processing content (the processing flow illustrated inFIG. 10) of the mirror adjusting process (S24) in the processing flowillustrated in FIG. 7 according to the first embodiment. Therefore,repeated detailed explanation will be omitted.

In order to move the display area 6 for the virtual image in verticaldirection, in the mirror adjusting process (S24) according to the firstembodiment, rotates the mirror 52 to a target angle. On the other hand,in the mirror state adjusting process (S346) according to the presentembodiment, as illustrated in FIG. 26, the ECU 21 may further move themirror 52, or rotate the mirror 52 around the z axis to adjust a stateof the mirror 52 to a target state by combining these. This makes itpossible to move the display state of the display area 6 itself for thevirtual image to an appropriate position in the vertical direction orrotate the display state in the horizontal direction.

Returning to FIG. 32, the ECU 21 repeats the series of processes atSteps S344 to S346 described above for each of the first and secondposture sensors 111 each of which detects the angle caused by jolting ofthe vehicle 2, and terminates the loop processing (S347).

The ECU 21 then calculates a display shift amount of the virtual imageto be displayed therein (that is, a moving amount and/or a rotatingamount with respect to the actual view in front of the vehicle 2) on thebasis of the display state of the display area 6 for the virtual imagewith respect to a current state of the mirror 52 (S348). The ECU 21executes, in the manner of software processing, adjustment to move thedisplay state of the virtual image in the display area 6 in the verticaldirection or rotate the display state in the horizontal direction on thebasis of the calculated display shift amount (S349), and terminates theseries of display state adjusting process.

As explained above, according to the head up display of the secondembodiment of the present invention, it is possible to flexibly suppressshaking of the virtual image with respect to the actual view ahead invarious kinds of complicated modes of jolting that the vehicle 2 occurs,and to display the virtual image so as to be superimposed onto anappropriate position. Further, by configuring the head up display todirectly acquire the adjusting amount of the display state of thevirtual image from the virtual image adjusting amount table on the basisof the angles detected by the first and second posture sensors 111without calculating it each time, it is possible to maintain a real-timeproperty in the adjustment of the display state of the virtual image,and reduce an uncomfortable feeling given to the driver 5.

As described above, the present invention made by the present inventorshas been explained specifically on the basis of the embodiments.However, the present invention is not limited to the embodiments, and itgoes without saying that the present invention may be modified intovarious forms without departing from the substance thereof. For example,the embodiments described above have been explained in detail forexplaining the present invention clearly. The present invention is notnecessarily limited to one that includes all configurations that havebeen explained. Further, a part of the configuration of one embodimentcan be replaced by a configuration of the other embodiment. Further, aconfiguration of the other embodiment can be added to a configuration ofone embodiment. Moreover, a part of the configuration of each of theembodiments can be added to the other configuration, deleted or replacedthereby.

INDUSTRIAL APPLICABILITY

The present invention is available for a head up display using AR.

REFERENCE SINGS LIST

1 . . . AR-HUD, 2 . . . vehicle, 3 . . . windshield, 4 . . . vehicleinformation, 5 . . . driver, 6 display area, 10 . . . vehicleinformation acquiring unit, 20 . . . controller, 21 . . . ECU, 22 . . .audio output unit, 23 . . . nonvolatile memory, 24 . . . memory, 25 . .. light source adjusting unit, 26 distortion correcting unit, 27 . . .display element driver, 28 . . . display distance adjusting unit, 29 . .. mirror adjusting unit, 30 . . . video display, 30 a . . . projector,30 b . . . LCD, 31 . . . light source, 32 . . . illumination opticalsystem, 33 . . . display element, 40 . . . display distance adjustingmechanism, 41 a . . . diffuser plate, 41 b . . . diffuser plate, 41 c .. . movable diffuser plate, 42 a . . . lens, 42 b . . . movable lens, 43a movable optical filter, 43 b . . . comb-shaped optical filter, 50 . .. mirror driver, 51 . . . mirror, 51 a . . . mirror, 51 b . . .photochromic mirror, 52 . . . mirror, 53 a to 53 c . . . mirror drivingshafts A to C, 54 a . . . supporting member, 54 b . . . mirror holdingmember, 60 . . . speaker, 101 . . . vehicle speed sensor, 102 . . .shift position sensor, 103 . . . handle steering angle sensor, 104 . . .headlight sensor, 105 . . . illuminance sensor, 106 . . . chromaticitysensor, 107 . . . distance measuring sensor, 108 . . . infrared sensor,109 . . . engine start sensor, 110 . . . acceleration sensor, 111 . . .posture sensor, 112 . . . temperature sensor, 113 . . . road-to-vehiclecommunication wireless receiver, 114 . . . vehicle-to-vehiclecommunication wireless receiver, 115 . . . camera (vehicle interior),116 . . . camera (vehicle exterior), 117 . . . GPS receiver, 118 . . .VICS receiver, 281 . . . functional liquid crystal film ON/OFFcontroller, 282 . . . lens moving unit, 283 . . . photochromic mirrorON/OFF controller, 284 . . . diffuser plate moving unit, 285 . . .optical filter moving unit, 401 . . . functional liquid crystal film,402 . . . lens moving mechanism, 403 . . . photochromic mirror, 404 . .. diffuser plate moving mechanism, 405 . . . optical filter movingmechanism

The invention claimed is:
 1. A head up display for a vehicle,comprising: a vehicle information acquiring unit configured to acquirevehicle information containing a lean of the vehicle in a pitchdirection, which is detected by a first posture sensor, and a lean in aroll direction, which is detected by a second posture sensor, the firstand second posture sensors being mounted on the vehicle; a controllerconfigured to control display of video on a basis of the vehicleinformation acquired by the vehicle information acquiring unit; a videodisplay configured to form the video in accordance with an instructionfrom the controller; a mirror configured to reflect the video formed bythe video display to project the video directly on to a windshieldhaving a curvature; and a mirror driver configured to change an angleand/or a position of the mirror on a basis of an instruction from thecontroller, wherein the controller is configured to adjust at least oneof a display state of a virtual image in a display area for the video orthe angle and/or the position of the mirror via the mirror driver on abasis of the lean in the pitch direction and/or the lean in the rolldirection in the vehicle information so that the virtual image issuperimposed onto a scene and the virtual image can be displayed to adriver, wherein the controller is further configured to control displayof the video to correct for distortion which occurs due to the curvatureof the windshield, wherein in a case where the display state of thevirtual image is adjusted and an adjusting amount is smaller than apredetermined value, the controller is configured to adjust the displaystate of the virtual image in the display area for the video on thebasis of the lean in the pitch direction and/or the lean in the rolldirection in the vehicle information, and wherein in a case where anadjusting amount is equal to or more than a predetermined value, thecontroller is configured to adjust the angle and/or the position of themirror via the mirror driver on the basis of the lean in the pitchdirection and/or the lean in the roll direction in the vehicleinformation, and further adjust the display state of the virtual imagein the display area for the video.
 2. The head up display according toclaim 1, wherein the controller is configured to restrict, in a casewhere the display state of the virtual image is adjusted and the virtualimage goes out from the display area, the adjustment to a state where atleast a part of the virtual image is displayed at an end portion of thedisplay area.
 3. The head up display according to claim 1, wherein atable is held in a nonvolatile memory, settings of a moving amount ofthe virtual image in a vertical direction with respect to each value ofthe lean in the pitch direction and settings of a rotating amount of thevirtual image in a horizontal direction with respect to each value ofthe lean in the roll direction in the vehicle information being held inthe table, and wherein the controller is configured to acquirecorresponding values of the moving amount of the virtual image in thevertical direction and/or the rotating amount of the virtual image inthe horizontal direction from the table on the basis of the lean in thepitch direction and/or the lean in the roll direction in the vehicleinformation.
 4. The head up display according to claim 3, wherein in thetable, each of the values of the moving amount of the virtual image inthe vertical direction and/or the rotating amount of the virtual imagein the horizontal direction is set to zero in a range in which each ofabsolute values of the lean in the pitch direction and/or the lean inthe roll direction is smaller than a predetermined value.
 5. The head updisplay according to claim 3, wherein the controller is configured tocalculate information related to the settings of the table on a basis ofa past performance of the vehicle information, and update content of thetable.
 6. A head up display for a vehicle, comprising: a vehicleinformation acquiring unit configured to acquire vehicle informationcontaining a lean of the vehicle in a pitch direction, which is detectedby a first posture sensor, and a lean in a roll direction, which isdetected by a second posture sensor, the first and second posturesensors being mounted on the vehicle; a controller configured to controldisplay of video on a basis of the vehicle information acquired by thevehicle information acquiring unit; a video display configured to formthe video in accordance with an instruction from the controller; amirror configured to reflect the video formed by the video display toproject the video to a windshield or combiner; and a mirror driverconfigured to change an angle and/or a position of the mirror on a basisof an instruction from the controller, wherein the controller isconfigured to adjust at least one of a display state of a virtual imagein a display area for the video or the angle and/or the position of themirror via the mirror driver on a basis of the lean in the pitchdirection and/or the lean in the roll direction in the vehicleinformation so that the virtual image is superimposed onto a scene andthe virtual image can be displayed to a driver, wherein in a case wherethe display state of the virtual image is adjusted and an adjustingamount is smaller than a predetermined value, the controller isconfigured to adjust the display state of the virtual image in thedisplay area for the video on the basis of the lean in the pitchdirection and/or the lean in the roll direction in the vehicleinformation, and in a case where an adjusting amount is equal to or morethan a predetermined value, the controller is configured to adjust theangle and/or the position of the mirror via the mirror driver on thebasis of the lean in the pitch direction and/or the lean in the rolldirection in the vehicle information, and further adjust the displaystate of the virtual image in the display area for the video.